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The  purpose  of  this  study  was  to  evaluate  perceptions  of  the  Defense  Finance  and  Accounting  Service  (DFAS) 
workforce  regarding  a  comparison  of  the  former  five-element  and  new  pass/fail  rating  systems.  The  first  phase  of 
this  project  conducted  in  November  1996,  primarily  assessed  workforce  perceptions  under  the  prior  system.  A 
survey  was  used  to  explore  employee  attitudes  toward  possible  changes  in  their  personnel  evaluation  program 
(Vandivier,  1996).  Five  questions  were  presented  for  their  responses  to  help  gauge  their  potential  acceptance  of 
the  proposed  changes.  The  questions  dealt  with  three  types  of  rating  plans  and  whether  employees  might  prefer 
certain  changes  in  performance  awards  and  critical  elements  relating  to  team  performance.  Employee  comments 
were  also  requested  in  the  survey.  Responses  indicated  that  the  prior  rating  system  of  five  elements  (Exceptional 
to  Unacceptable)  as  least  desirable  in  terms  of  first  choice,  the  three-level  plan  (Exceptional,  Fully  Successful 
and  Unacceptable)  seemed  marginally  desirable,  and  the  basic  "pass/fail"  rating  plan  was  given  the  most  initial 
first  choices.  (Vandivier,  Bridges,  &  Waldkoetter,  1996). 

METHOD 

The  second  phase,  focus  of  this  study,  assessed  perceptions  under  the  new  pass/fail  system  (Vandivier,  1998). 
Similar  survey  forms  as  administered  to  the  workforce  during  November  1 996,  were  prepared  and  distributed 
from  late  1997  to  January  1998.  These  surveys  were  sent  to  and  administered  within  the  selected  sample  of 
directorates.  Both  surveys  (prior  and  current  study)  included  questionnaire  items  to  be  completed  by  employees, 
and  supervisors  as  employees,  and  items  completed  only  by  supervisors.  The  second  phase  survey  included  two 
additional  questionnaire  items  (23  and  41)  asking  about  their  preference  for  the  new  versus  prior  systems  with 
item  23  for  employees  and  41  for  supervisors.  Also,  the  present  study’s  survey  was  adapted  so  respondents  could 
indicate  their  last  performance  rating  under  the  new  pass/fail  system, rather  than  the  prior  five-element 
(Exceptional  to  Unacceptable)  system.  The  parallel  structure  of  numerous  questionnaire  items  allowed 
comparisons  of  employee  and  supervisor  perceptions. 

Structure  of  the  performance  management  post  survey  (second  phase)  consisted  of  six  common  background 
items,  17  scaled  items  covering  performance  planning,  performance  feedback/counseling,  and  performance 
appraisal  and  awards  for  employees  and  supervisor-employees,  and  four  supervisory  only  background  items, 
with  14  scaled  items  covering  content  areas  indicated  above.  Both  sets  of  scaled  items  asked  respondents  to 
indicate  the  extent  to  which  they  "agreed  or  disagreed"  to  given  statements  on  a  five-point  scale  from  "strongly 
disagree"  to  "strongly  agree"  with  a  parallel  column  for  "don’t  know".  A  total  of  1500  survey  forms  were 
distributed  through  the  DFAS  Center  directorates  with  a  total  of  746  (49.7%)  collected  suggesting  a  high 
confidence  level  (95%)  in  the  overall  results  for  this  post  survey. 

RESULTS  AND  DISCUSSION 

A  greater  number  of  respondents  were  in  grades  1 1  and  12  (41 .5%),  followed  by  those  in  grades  6  through  10 
(22.5%),  with  grades  1  through  5  (19.1%),  grades  13  or  above  (16.3%),  and  others  (0.7%).  Respondents  indicated 
a  range  of  years  in  government  service  with  16  to  25  years  (31.1%),  followed  by  11  to  15  (20.6%),  6  to  10 
(18.6%),  with  26  or  more  (16.8%),  and  less  than  a  year  to  5  (12.9%).  A  total  of  88.1%  of  the  collected  survey 
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forms  were  from  employees  and  1 1 .9%  from  supervisors.  Nearly  all  respondents  (95%)  indicated  a  performance 
rating  of  "pass"  on  the  new  "pass/fail"  rating  system  as  shown  by  background  item  4.  Any  "don’t  know" 
responses  on  the  five-point  scaled  items  were  dismissed  for  possible  analytical  use.  Analysis  consisted  of 
calculating  averages  (means)  and  making  comparisons  across  various  groupings  providing  the  following  most 
salient  results. 

The  workforce  rating  yielded  a  mean  of  2.91  (N=720),  regarding  their  preference  for  the  "pass/fail"  system, 
which  rather  clearly  suggests  an  "undecided"  preference,  with  3  being  the  five-point  scale  midpoint  for 
"undecided."  Supervisors  responding  too,  as  employees,  tended  to  bias  the  workforce  rating,  since  the 
supervisors  had  a  stronger  preference  for  the  "pass/fail"  system,  yielding  a  mean  rating  of  4.07  (N=65),  which 
barely  exceeded  a  4  rating  for  "agree."  Workforce  preference  for  the  "pass/fail"  system  recorded  430  ratings  in 
the  "undecided"  to  "strongly  disagree"  range,  which  surely  suggests  a  negative  posture  even  with  the  inclusion  of 
the  more  positive  supervisory  ratings. 

While  75%  of  the  supervisors  indicated  their  probable  preference  for  the  "pass/fail"  system,  employees  only 
reflected  their  lesser  preference  with  just  40%  rating  in  the  "agree"  to  "strongly  agree"  (4  and  5  point)  range.  The 
above  differences  in  ratings  and  percentages  offer,  to  say  the  least,  very  practical  significance  for  further  analysis 
and  review,  and  likely  reflect  pointed  significant  differences  that  may  raise  further  issues  in  working  to 
implement  the  "pass/fail"  system  most  effectively  in  the  overall  performance  management  process. 

Analysis  of  the  results  indicated  no  significant  differences  in  the  overall  means  for  all  rated  items  from  pre-  to 
post-testing.  Employees  yielded  mean  rating  values  of  2.96  to  3.25  from  the  prior  survey  to  the  current  study 
with  supervisors  showing  3.61  to  3.87,  respectively  (Vandivier,  1998).  These  rating  values  would  seem  to 
illustrate  a  cautious  perception  of  the  utility  of  the  new  "pass/fail"  system  in  its  early  implementation.  Findings 
further  suggest  that  relatively  minor  changes  have  occurred  in  the  performance  management  processes  under  the 
new  "pass/fail"  system.  It  has  been  observed  previously  that  the  personnel  evaluation  and  management  program 
must  clearly  delineate  the  effectiveness  of  any  rating  process  and  the  implementing  management  guidelines  or 
prepare  to  address  any  contradictions  with  timely  and  conclusive  mediation  (Ross  &  Wieland,  1996). 

Estimated  times  for  performances  planning  comparing  employees  and  supervisors,  and  for  their  performance 
feedback/counseling  provided  some  interesting  observations.  It  was  noted  that  employees  (47%)  received  "less 
that  30  minutes"  performance  planning  time  with  supervisors  (12%)  responding  also  in  that  choice.  This  does  not 
necessarily  imply  inadequate  planning  time,  since  employees  (53%)  otherwise  received  "30  minutes  or  more  in 
performance  planning  with  supervisors  (88%)  obtaining  a  similar  balance  of  planning  time.  There  is  need  for 
further  data  analysis  to  determine  what  the  optimal  criteria  may  be  for  defining  what  performance  planning  times 
are  needed  and  feasible.  Although  unlikely,  it  seems  the  above  planning  time  estimates  may  suggest  an  overall 
normative  condition,  but  which  will  vary  somewhat  among  participating  directorates  and  workforce  operations. 

Further  the  estimated  times  comparing  employees  and  supervisors  for  their  respective  performance 
feedback/counseling,  revealed  employees  (43%)  received  "less  that  30  minutes"  with  7%  of  the  supervisors 
falling  in  this  estimated  range.  Again,  there  is  no  evidence  to  question  this  operational  estimate  with  employees 
(57%)  and  supervisors  (93%)  receiving  "30  minutes  or  more"  for  their  estimated  performance 
feedback/counseling  times.  Here  it  must  be  observed  also  that  this  operational  estimate  may  be  relatively 
accurate,  but  data  analysis  must  go  beyond  this  stage  to  develop  what  required  and  preferred  feedback/counseling 
times  are  needed  and  feasible  as  well.  Perhaps  in  performance  feedback/counseling  reviews,  this  issue  can  be 
reconciled  by  analyzing  the  differing  roles  of  supervisors  and  employees  to  propose  operationally  feasible  times 
for  this  critical  function.  It  should  be  practical  to  have  supervisors  and  performance  management  operators 
express  necessary  standards  for  the  personnel  evaluation  program  based  on  their  higher  levels  of  task  experience. 
Then,  employees  may  be  able  to  express  criterion  and  more  accurate  estimates  of  how  performance 
feedback/counseling  and  personnel  evaluation  tasks  can  be  operationally  performed  (Richman  &  Quinones, 
1996). 

In  closing,  a  number  of  steps  are  recommended  to  help  the  management  leadership  and  the  Human  Resource 
Directorate  adopt  constructive  procedures  to  review  the  viability  of  the  "pass /fail"  rating  system  (V andivier, 
1998).  First  the  findings  should  be  effectively  distributed  to  provide  applicable  information  to  the  workforce,  and 
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effort  should  be  made  to  ensure  more  extensive  participation  for  any  future  survey  efforts.  Additionally,  there 
may  be  sufficient  reason  to  examine  the  psychological  climate  in  regard  to  the  workforce’s  work  situations  to 
present  the  most  positive  factors  in  options  for  changes  in  the  personnel  evaluation  program  (Brown  &  Leigh, 


Next,  a  summary  review  of  "high  and  low"  rated  items  should  be  made  to  create  a  total  quality  management  plan 
for  strategic  action.  Then,  set  objectives  which  can  measurably  improve  the  performance  management  system 
regarding  all  aspects  of  "performance  planning,  performance  feedback/counseling,  and  performance  appraisal 
and  awards."  Lastly,  the  principal  study  analyst  should  be  retained  in  a  consulting  and  advisory  capacity  to 
clarify  existing  information  and  participate  in  any  changes  to  modify  or  redesign  the  newly  implemented 
"pass/fail"  system. 
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